How Rare is Rare?

Large Deviations is a branch of probability theory that deals with rare events. It is a very active field with applications to several areas including statistics, communication networks, computing, statistical physics, and mathematical finance. (For an example of a rare event, consider the problem of species sampling: suppose that one wishes to determine the species of fish native to a body of water and that, after repeated sampling, one identifies a certain number of species. Suppose that n trials have been conducted. The problem is to estimate the probability, for larger n, that one would observe a new species on the next trial.) Roughly speaking, one may know, in a given situation, that the probabilities of certain events tend to zero as some parameter goes to ∞ (or zero). The question arises as to how fast the probabilities converge. Large deviation techniques enable one to find the rate of convergence. Central to the subject is the large deviation principle (LDP), enunciated by S.R.S. Varadhan in 1966. A natural context for stating the LDP is that of Borel probability measures on a Polish (i.e., a complete, separable metric) space. The reason is that the space of Borel probability measures on a Polish space can be metrized in such a way